Counterfactual Optimism: Rate Optimal Regret for Stochastic Contextual MDPs

We present the UC$^3$RL algorithm for regret minimization in Stochastic Contextual MDPs (CMDPs). The algorithm operates under the minimal assumptions of realizable function class, and access to offline least squares and log loss regression oracles. Our algorithm is efficient (assuming efficient offline regression oracles) and enjoys an $\widetilde{O}(H^3 \sqrt{T |S| |A|(\log (|\mathcal{F}|/\delta) + \log (|\mathcal{P}|/ \delta) )})$ regret guarantee, with $T$ being the number of episodes, $S$ the state space, $A$ the action space, $H$ the horizon, and $\mathcal{P}$ and $\mathcal{F}$ are finite function classes, used to approximate the context-dependent dynamics and rewards, respectively. To the best of our knowledge, our algorithm is the first efficient and rate-optimal regret minimization algorithm for CMDPs, which operates under the general offline function approximation setting

Efficient Rate Optimal Regret for Adversarial Contextual MDPs Using Online Function Approximation

We present the OMG-CMDP! algorithm for regret minimization in adversarial Contextual MDPs. The algorithm operates under the minimal assumptions of realizable function class and access to online least squares and log loss regression oracles. Our algorithm is efficient (assuming efficient online regression oracles), simple and robust to approximation errors. It enjoys an $\widetilde{O}(H^{2.5} \sqrt{ T|S||A| ( \mathcal{R}(\mathcal{O}) + H \log(\delta^{-1}) )})$ regret guarantee, with $T$ being the number of episodes, $S$ the state space, $A$ the action space, $H$ the horizon and $\mathcal{R}(\mathcal{O}) = \mathcal{R}(\mathcal{O}_{\mathrm{sq}}^\mathcal{F}) + \mathcal{R}(\mathcal{O}_{\mathrm{log}}^\mathcal{P})$ is the sum of the regression oracles' regret, used to approximate the context-dependent rewards and dynamics, respectively. To the best of our knowledge, our algorithm is the first efficient rate optimal regret minimization algorithm for adversarial CMDPs that operates under the minimal standard assumption of online function approximation

RadArnomaly: Protecting Radar Systems from Data Manipulation Attacks

Radar systems are mainly used for tracking aircraft, missiles, satellites, and watercraft. In many cases, information regarding the objects detected by the radar system is sent to, and used by, a peripheral consuming system, such as a missile system or a graphical user interface used by an operator. Those systems process the data stream and make real-time, operational decisions based on the data received. Given this, the reliability and availability of information provided by radar systems has grown in importance. Although the field of cyber security has been continuously evolving, no prior research has focused on anomaly detection in radar systems. In this paper, we present a deep learning-based method for detecting anomalies in radar system data streams. We propose a novel technique which learns the correlation between numerical features and an embedding representation of categorical features in an unsupervised manner. The proposed technique, which allows the detection of malicious manipulation of critical fields in the data stream, is complemented by a timing-interval anomaly detection mechanism proposed for the detection of message dropping attempts. Real radar system data is used to evaluate the proposed method. Our experiments demonstrate the method's high detection accuracy on a variety of data stream manipulation attacks (average detection rate of 88% with 1.59% false alarms) and message dropping attacks (average detection rate of 92% with 2.2% false alarms)

In Planta Colonization and Role of T6SS in Two Rice Kosakonia Endophytes.

Endophytes live inside plants and are often beneficial. Kosakonia is a novel bacterial genus that includes many diazotrophic plant-associated isolates. Plant–bacteria studies on two rice endophytic Kosakonia beneficial strains were performed, including comparative genomics, secretome profiling, in planta tests, and a field release trial. The strains are efficient rhizoplane and root endosphere colonizers and localized in the root cortex. Secretomics revealed 144 putative secreted proteins, including type VI secretory system (T6SS) proteins. A Kosakonia T6SS genomic knock-out mutant showed a significant decrease in rhizoplane and endosphere colonization ability. A field trial using rice seed inoculated with Kosakonia spp. showed no effect on plant growth promotion upon nitrogen stress and microbiome studies revealed that Kosakonia spp. were significantly more present in the inoculated rice. Comparative genomics indicated that several protein domains were enriched in plant-associated Kosakonia spp. This study highlights that Kosakonia is an important, recently classified genus involved in plant–bacteria interaction

DropCompute: simple and more robust distributed synchronous training via compute variance reduction

Background: Distributed training is essential for large scale training of deep neural networks (DNNs). The dominant methods for large scale DNN training are synchronous (e.g. All-Reduce), but these require waiting for all workers in each step. Thus, these methods are limited by the delays caused by straggling workers. Results: We study a typical scenario in which workers are straggling due to variability in compute time. We find an analytical relation between compute time properties and scalability limitations, caused by such straggling workers. With these findings, we propose a simple yet effective decentralized method to reduce the variation among workers and thus improve the robustness of synchronous training. This method can be integrated with the widely used All-Reduce. Our findings are validated on large-scale training tasks using 200 Gaudi Accelerators.Comment: https://github.com/paper-submissions/dropcomput

Links between Main Frequencies of Established Rotating Stall and Rotational Frequencies and/or Blade Passing Frequencies

The ratios between the main frequency of rotating stall and rotational frequency may be considered in the form of exact ratios of small natural numbers if the pressure signals in compressors during rotating stall include the rotor rotation frequency component. During rotating stall in compressors with good rotor balancing (with absence of the rotational frequency component in the frequency characteristics of pressure signals), these ratios between the main frequency of rotating stall and rotational frequency are or are not in the form of ratios of small natural numbers. The experimentally received characteristics of power spectral density of pressure signals also show the presence of components with combinations of blade passing frequency and different harmonics of main rotating stall frequency